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METHOD AND APPARATUS FOR THREE DIMENSIONAL INSPECTION OF 

ELECTRONIC COMPONENTS 

CROSS-REFERENCE TO RELATED APPLICATIONS 

This application is a continuation-in-part of pending U.S. 
application no. 09/351,892 filed 07/13/1999, entitled "Method and 
Apparatus for Three Dimensional Inspection of Electronic 
Components," incorporated by reference herein, that is a 
continuation-in-part of U.S. Patent No. 6,072,898 issued June 6, 
2000, entitled "Method and Apparatus for Three Dimensional 
Inspection of Electronic Components," incorporated by reference 
herein . 

NOTICE RE COPYRIGHT 

A portion of the disclosure of this patent document contains 
material which is subject to copyright protection. The copyright 
owner has no objection to the facsimile reproduction by anyone of 
the patent disclosure, as it appears in the Patent and Trademark 
Office patent files or records, but otherwise reserves all 
copyright rights whatsoever. 

FIELD OF THE INVENTION 

This invention relates to a method and apparatus for three 
dimensional inspection and, more particularly, a method and 
apparatus for three dimensional inspection of solder balls on 
ball grid arrays and solder bumps on wafer and die, and to a 
calibration method. 

BACKGROUND OF THE INVENTION 
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Prior art three dimensional inspection systems have involved 
laser range finding technology, moire interf erometry, structured 
light patterns or two cameras. The laser range finding method 
directs a focused laser beam onto the Ball Grid Array, BGA, and 
5 detects the reflected beam with a sensor. Elements of the BGA 
are determined in the X, Y and Z dimensions utilizing a 
triangulation method. This method requires a large number of 
measurement samples to determine the dimensions of the BGA 
resulting in longer inspection times. This method also suffers 

S-Jj 

iQ 10 from specular reflections from the smooth surfaces of the solder 

13 

*p balls resulting in erroneous data, 

ft] Moire interf erometry utilizes the interference of light 

m waves generated by a diffraction grating to produce a pattern of 

P dark contours on the surface of the BGA. These contours are of 

3S33 

111 15 known distance in the Z dimension from the diffraction grating, 
f y 

,i By counting the number of contours from one point on the BGA- to 
°^ another point on the BGA, the distance in the Z dimension between 

the two points can be determined. This method suffers from the 
problem of low contrast contour lines resulting in missed 
20 counting of the number of contours and resulting in erroneous 

data. This method also suffers from the contour lines merging at 
surfaces with steep slopes, such as the sides of the balls on the 
BGA, resulting in an incorrect count of the number of contours 
and resulting in erroneous data. 
25 Structured light systems project precise bands of light onto 
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the part to be inspected. The deviation of the light band from a 
straight line is proportional to the distance from a reference 
surface. The light bands are moved across the part, or 
alternately the part is moved with respect to the light bands, 
5 and successive images are acquired. The maximum deviation of the 
light band indicates the maximum height of a ball. This method 
suffers from specular reflections due to the highly focused 
nature of the light bands resulting in erroneous data. This 
method further suffers from increased inspection times due to the 

O 

'0 10 number of images required. 

1 3 

Two camera systems utilize one camera to view the BGA device 
flj in the normal direction to determine X and Y dimensions and the 
jij second camera to view the far edges of the balls from an angle. 
jU The two images are combined to determine the apparent height of 
Py 15 each ball in the Z dimension utilizing a triangulation method. 
:2 This method suffers from the need for a higher angle of view of 
[ ^ the ball from the second camera resulting in looking at a point 
significantly below the top of the ball for BGA 1 s having fine 
pitch. This method also suffers from limited depth of focus for 
20 the second camera limiting the size of BGA 1 s that can be 

inspected. This system can only inspect BGA T s and not other 
device types such as gullwing and J lead devices. 

The prior art does not provide two separate and opposite 
side views permitting larger BGA 1 s to be inspected or nonlinear 
25 optics to enhance the separation between adjacent ball images in 
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the side perspective view. 

It is therefore a motivation of the invention to improve the 
accuracy of the measurements, the speed of the measurements, the 
ability to measure all sizes and pitches of BGA's and to measure 
5 other devices including gullwing and J lead parts in a single 
system, 

SUMMARY OF THE INVENTION 

The invention provides an apparatus for three dimensional 
inspection of an electronic part, wherein the apparatus is 

D 

*3 10 calibrated using a precision pattern mask with dot patterns 

IB 

>fi deposited on a calibration transparent reticle, the apparatus for 
FU three dimensional inspection of an electronic part comprising a 

: . ! 

jtj camera and an illuminator for imaging the electronic part, the 
O camera being positioned to obtain a first view of the electronic 

III 15 part, a means for light reflection positioned to reflect a 
£ different view of the electronic part into the camera, wherein 

if-™? 

^ the camera provides an image of the electronic part having 

differing views of the electronic part, and a means for image 
processing the image of the electronic part that applies 
20 calculations on the differing views of the image to calculate a 
three dimensional position of at least one portion of the 
electronic part. 

The invention further comprises a ring light. The means for 
light reflection could further comprise a mirror, a prism, or a 
25 curved mirror. The electronic part may be a ball grid array, 
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balls on a wafer, or balls on a die. 

The means for imaging provides the image to a frame grabber 
board. The frame grabber board provides an image data output to 
a processor to perform a three dimensional inspection of the 
5 part . 

The apparatus may further comprise a nonlinear optical 
element to magnify the second image in one dimension. In the 
apparatus a maximum depth of focus of a side perspective view 
allows for a fixed focus system to inspect larger electronic 
*fi 10 parts, with one perspective view imaging one portion of the 
<jP electronic part and a second perspective view imaging a second 
f|j portion of the electronic part. Also, in the apparatus a maximum 

y 

III depth of focus of a side perspective view includes an area of the 
P electronic part including a center row of balls. Furthermore, all 

m 15 of the balls on the electronic part may be in focus resulting in 

M 

i«i two perspective views for each ball. 

The invention comprises a means for inspecting gullwing and 
J lead devices . 

The invention further provides a method for three 
20 dimensional inspection of a lead on a part, the method comprising 
the steps of using a camera to receive an image of the lead, 
transmitting the image of the lead to a frame grabber, providing 
fixed optical elements to obtain a side perspective view of the 
lead, transmitting the side perspective view of the lead to the 
25 frame grabber, operating a processor to send a command to the 
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frame grabber to acquire images of pixel values from the camera, 
and processing the pixel values with the processor to calculate a 
three dimensional position of the lead. State values may be 
determined from the part itself. 
5 The lead may be a curved surface lead, a ball, a ball grid 

array, a formed wire, a stamped metal form or similar object that 
can be imaged from two separate directions. 

The processor processes the pixel values to find a rotation, 
an X placement value and a Y placement value of the part relative 

O 

*fl 10 to world X and Y coordinates by finding points on four sides of 
130 

the part . 

IU The invention further provides the steps of using a part 

S I 

ijy 

rU definition file that contains measurement values for an ideal 

El 

g part, calculating . an expected position for each lead of the part 
jy 15 for a bottom view using the measurement values from the part 
™ definition file and the X placement value and Y placement value. 

^ The invention further provides the step of using a search 

procedure on the image data to locate the lead. 

The invention further provides the step of determining a 
20 lead center location and a lead diameter in pixels and storing 
the lead center location and lead diameter in memory. 

The invention further provides the step of calculating an 
expected position of a center of each lead in both side 
perspective views in the image using a known position of each 
25 side view from calibration. 



The invention further provides the step of using a subpixel 
edge detection method to locate a reference point on each lead. 

The invention further provides the step of converting the 
pixel values into world locations by using pixel values and 
5 parameters determined during calibration wherein the world 

locations represent physical locations of the lead with respect 
to world coordinates defined during calibration. 

The invention further provides for the calculation of a Z 
height of each lead in world coordinates in pixel values by 

□ 

*S 10 combining a location of a center of a lead from a bottom view 

m 

■S with a reference point of the same lead from a side perspective 
•P 

ly view. 

jlj The invention further provides the step of converting the 

B world values to part values using the rotation, the X placement 
15 value and the Y placement value to define part coordinates for 
the ideal part where the part values represent physical 
dimensions of the lead including lead diameter, lead center 
location in X part and Y part coordinates and lead height in Z 
world coordinates. 
20 The invention further provides the step of comparing ideal 

values defined in the part file to calculate deviation values 
that represent a deviation of the center of the lead from its 
ideal location. The deviation values may include lead diameter 
in several orientations with respect to the X placement value and 
25 Y placement value, lead center in the X direction, Y direction 



Si 
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and radial direction, lead pitch in the X direction and Y 
direction and missing and deformed leads, further comprising the 
step of calculating the Z dimension of the lead with respect to 
the seating plane based on the Z world data. 
5 The invention further provides the step of comparing the 

deviation values to predetermined tolerance values with respect 
to an ideal part as defined in the part definition file to 
provide a lead inspection result. 

BRIEF DESCRIPTION OF THE DRAWINGS 

o 

*C 10 To illustrate this invention, preferred embodiments will be 

m 

•f described herein with reference to the accompanying drawings. 

S3 

PJ Figure 1A shows the apparatus of the invention for system 

!y calibration. 

O Figures 1B1, 1B2 and 1B3 show an example calibration pattern 

5 15 and example images of the calibration pattern acquired by the 

H 

q system. 

'** Figure 2A shows a flow chart of a method of the invention 

used for calibration of the bottom view. 

Figure 2B shows a flow chart of a method of the invention 
20 used for determining the state values, and the X and Y world 
coordinates, of the bottom view of the system. 

Figure 2C shows a flow chart of a method of the invention 
used for calibration of the side perspective views. 

Figure 2D shows a flow chart of a method of the invention 
25 used for determining the state values of the side perspective 
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views of the system. 

Figure 2E shows the relationship of a side perspective angle 
to the ratio of the perspective dimensions to the non-perspective 
dimensions . 

5 Figure 2F shows a bottom view and a side perspective view of 

precision dots used in the method for determining a side 
perspective view angle . 

Figure 3A shows the apparatus of the invention for part 
inspection. 

'■B 10 Figures 3B1, 3B2 and 3B3 show example images of a part 

>P acquired by the system. 

fy Figure 4 shows a method of the invention for the three 

y 

fU dimensional inspection of balls on a ball grid array. 

i\ 

p Figures 5A and 5B together show a flow chart of the three 

tfco 

iy 15 dimensional inspection method of the invention. 

r ~ Figures 6A and 6B show an example ball of a ball grid array 

^ and associated geometry used in a method of the invention for 
determining the Z position of the ball. 

Figure 7A shows one example of an image used in the 
20 grayscale blob method of the invention. 

Figure 7B shows one example of an image used with the method 
of the invention to perform a subpixel measurement of the ball 
reference point. 

Figure 8A shows a side perspective image of the calibration 
25 pattern magnified in one dimension. 
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Figure 8B shows a side perspective image of the balls on a 
BGA, magnified in one dimension. 

Figure 9 shows an apparatus for presenting a BGA for 
inspection. 

5 Figures 10A and 10B show an example ball of a ball grid 

array with associated geometry as used with a method of the 
invention for determining the Z position of a ball using two side 
perspective views. 

Figure 11A shows the apparatus of the invention for system 
M3 10 calibration, utilizing a single side perspective view. 

m 

:p Figures 11B1, 11B2 and 11B3 show an example calibration 

-fa 

ill pattern and example images of a calibration pattern acquired by 

uj 

ITJ the system, utilizing a single side perspective view, of the 
n invention. 

ifj 15 , Figure 12A shows the apparatus of the invention for ball 
( 2 inspection utilizing a single side perspective view. 
|( * Figures 12B1, 12B2 and 12B3 show an example ball grid array 

and example images of the ball grid array for three dimensional 
inspection, utilizing a single side perspective view. 
20 Figure 13 shows the apparatus of the invention for the three 

dimensional inspection of ball grid array devices, gullwing 
devices and J lead devices. 

Figure 14 shows the apparatus of the invention for the three 
dimensional inspection of parts utilizing three cameras. 
25 Figure 15 shows the apparatus of the invention configured 
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with a calibration reticle 1020 for use during calibration of the 
state values of the system. 

Figures 16A and 16B show an example calibration pattern and 
example images of the calibration pattern acquired by the single 
5 camera system. 

Figure 17 shows the apparatus of the invention configured 
with a part 1040 to be inspected by the system. 

Figures 18A and 18B show a part and example images of a part 
acquired by the system. 

O 

ifi 10 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

m 

: £: In one embodiment of the invention, the method and apparatus 

jfj disclosed herein is a method and apparatus for calibrating the 
fy system by placing a pattern of calibration dots of known spacing 
H and size on the bottom plane of a calibration reticle. From the 
jfl 15 precision dots the missing state values of the system are 
m determined allowing for three dimensional inspection of balls on 
ball grid array devices, BGA devices or balls on wafers or balls 
on die. In one embodiment of the invention the system may also 
inspect gullwing and J lead devices as well as ball grid arrays. 
20 Refer now to Figure 1A which shows the apparatus of the 

invention configured with a calibration reticle for use during 
calibration of the state values of the system. The apparatus 
obtains what is known as a bottom image 50 of the calibration 
reticle 20. To take the bottom image 50 the apparatus includes a 
25 camera 10 with a lens 11 and calibration reticle 20 with a 
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calibration pattern 22 on the bottom surface. The calibration 
pattern 22 on the reticle 20 comprises precision dots 24, The 
camera 10 is located below the central part of the calibration 
reticle 2 0 to receive an image 50 described in conjunction with 
5 Figure IB, In one embodiment the camera 10 comprises an image 
sensor. The image sensor may be a charged coupled device array. 

The camera 10 is connected to a frame grabber board 12 to 
receive the image 50. The frame grabber board 12 provides an 
imaqe data output to a processor 13 to perform a two dimensional 

O 

^ 10 calibration as described in conjunction with Figure 2A. The 
OQ 

<F processor 13 may store an image in memory 14. The apparatus of 
HJ the invention obtains an image of a pair of side perspective 

y 

111 views and includes using a camera 15 with a lens 16 and a 

;: 

Q calibration reticle 20. The camera 15 is located to receive an 
tj 15 image 60, comprising a pair of side perspective views, described 



in conjunction with Figure IB. Fixed optical elements 30, 32 and 
38 provide a first side perspective view and fixed optical 
elements 34, 36, 38 for a second side perspective view. The 
fixed optical elements 30, 32, 34, 36 and 38 may be mirrors or 

20 prisms. As will be appreciated by those skilled in the art 

additional optical elements may be incorporated. The camera 15 
is connected to a frame grabber board 17 to receive the image 60. 

The frame grabber board 17 provides an image data output to a 
processor 13 to perform a two dimensional inspection as 

25 described in conjunction with Figure 2B. The processor 13 may 
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store an image in memory 14. In one embodiment of the invention, 
the apparatus may contain a nonlinear optical element 39 to 
magnify the side perspective image 60 in one dimension as shown 
in Figure 8A. In another embodiment of the invention optical 
5 element 38 may be a nonlinear element. The nonlinear optical 
elements 38 and 39 may be a curved mirror or a lens. 

Figures 1B1, 1B2 and 1B3 show an example image 50 from 
camera 10 and an example image 60 from camera 15 acquired by the 
system. The image 50, a bottom view of dot pattern 22, shows 

13 

k 0 io dots 52 acquired by camera 10. The dot pattern contains 

m 

*F precision dots 24 of known dimensions and spacing. The precision 
I1J dots 24 are located on the bottom surface of the calibration 
ilj reticle 20. The image 60 shows two side perspective views of the 
□ dot pattern 22. A first side perspective view in image 60 

5 15 contains images 62 of dots 24 and is obtained by the reflection 
3 of the image of the calibration reticle dot pattern 22 off of 
"* fixed optical elements 30, 32 and 38 into camera 15. A second 

side perspective view in image 60 contains images 66 of dots 24 
and is obtained by the reflection of the image of the calibration 
20 reticle dot pattern 22 off of fixed optical elements 34, 36 and 
38 into camera 15. 

Optical element 36 is positioned to adjust the optical path 
length of a second side perspective view to equal the optical 
path length of a first side perspective view. Those skilled in 
25 the art will realize that any number of perspective views can be 



# # 

14 

utilized by the invention. In one embodiment of the invention, 
the maximum depth of focus of a side perspective view includes an 
area of the reticle including the center row of dots. This 
allows for a fixed focus system to inspect larger parts, with one 
5 perspective view imaging half of the part and the second 
perspective view imaging the other half of the part. 

Figure 2A shows a flow diagram for the calibration of the 
bottom view of the system. The method starts in step 101 by 
providing a transparent reticle 20 having a bottom surface 

"0 10 containing a dot pattern 22, comprising precision dots 24 of 

IS 

'P known dimensions and spacing. The method in step 102 provides a 

ill camera 10 located beneath the transparent reticle 20 to receive 

W 

iU an image 50. In step 103 the processor 13 sends a command to a 

□ frame grabber 12 to acquire an image 50, comprising pixel values 

■I* 

jy 15 from the camera 10. The method then proceeds to step 104 and 

□ processes the pixel values with a processor 13. 

Figure 2B shows a flow diagram for determining the state 
values of the bottom view of the system. In step 111 the method 
begins by finding the dots 52 in image 50, corresponding to the 

20 calibration dots 24. The processor finds a dimension and 
position for each dot visible in image 50 in subpixel values 
using well known grayscale methods and stores these values in 
memory 14. By comparing these results to known values stored in 
memory, the processor calculates the missing state values for the 

25 bottom calibration in steps 112 and 113. In step 112 the 
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processor 13 calculates the optical distortion of lens 11 and the 

camera roll angle with respect to the dot pattern 22. Step 113 

calculates the pixel width and pixel height by comparing the 

subpixel data of dots 52 with the known dimensions of the 

5 precision dot pattern 22. The pixel aspect ratio is determined 

from the pixel width and pixel height. In step 114 the processor 

defines the X and Y world coordinates and the Z=0 plane from the 

image 50 of the precision dot pattern 22. The processor then 

stores these results in memory. These results provide conversion 

*^ 10 factors for use during analysis to convert pixel values to world 
!£} 

s i* values. 

fU Figure 2C shows a flow diagram for the calibration of the 

w 

TU side perspective views of the system. The method starts in step 
□ 121 by providing a transparent reticle 20 having a bottom surface 

e f=" 

iU 15 containing a dot pattern 22, comprising precision dots 24 of 
Q known dimensions and spacing. The method in step 122 provides 
fixed optical elements 30, 32, 34, 36 and 38 to reflect two 
perspective images of the precision dot pattern 22 into camera 
15. The method in step 123 provides a camera 15 located to 
20 receive an image 60. In step 124 the processor 13 sends a 

command to a frame grabber 12 to acquire an image 60, comprising 
pixel values from the camera 15. The method then proceeds to 
step 125 and processes the pixel values with a processor 13. 
Figure 2D shows a flow diagram for determining the state 
25 values of the side perspective views of the system. In step 131 
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the method begins by finding dots 62 in image 60, corresponding 
to the calibration dots 24. The processor finds a dimension and 
position for each dot visible, comprising the group of dots 62, 
in image 60 for a first side perspective view in subpixel values 
5 and stores these values in memory 14, By comparing these results 
to known values stored in memory, the processor calculates the 
missing state values for a side perspective view, comprising the 
group of dots 62, in steps 132 and 133. In step 132 the 
processor 13 calculates the optical distortion of lens 16 and the 

□ 

*y 10 camera roll angle with respect to the dot pattern 22. In step 
=F 133 the processor 13 calculates the pixel width and pixel height 

!u by comparing the subpixel data of dots 62 with the known 

W 

nj dimensions of the precision dots 24. The pixel aspect ratio is 

SI 

O determined from the pixel width and pixel height. In step 134 
Hf 15 the processor defines the X and Y world coordinates and the Z=0 
^ plane from the dots 62 in image 60 of the dot pattern 22. The 
1=53 processor then stores these results in memory. These results 

provide conversion factors for use during analysis to convert 

pixel values to world values. In step 135 the method of the 
20 invention computes the side view angle. In step 136 the method 

is repeated for a second side perspective view using the dots 66 

in image 60. 

Figure 2E shows the relationship of a side perspective angle 
to the ratio of the perspective dimension to the non-perspective 
25 dimension. Ray 171, 172, and 173 defining point 181 is parallel 



to ray 174, 175 and 176 defining point 182, Point 181 and point 
182 lie on a plane 170 parallel to a plane 180. The intersection 
of ray 175 and ray 176 define point 186. The intersection of ray 
176 and ray 172 define point 184, The intersection of ray 173 
and ray 172 define point 187. The intersection of ray 174 and ray 
172 define point 183. The reflecting plane 179 intersecting plane 
180 at an angle D is defined by ray 172 and ray 175 and the law 
of reflectance. Ray 172 and ray 175 intersect plane 170 at an 
angle 177. Referring to Figure 2E it can be shown: 



tan 0 



C_ 

Db 



C 



L 



sin A sin A 



Therefore : C = L 



a Ds Ds 
L C 

COS0 



Substituting: 
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Ds 

D B D b cos0 

(tan0 )(cos0 ) = — =sin <9 
D B 



ill 



# = arcsin 



Figure 2F shows a bottom view and a side perspective view of 
precision dots used in the method for determining a side 
perspective view angle 177 as shown in Figure 2E of the system. A 
10 bottom view image 200 comprising precision dots 201, 202 and 203 
of known spacing and dimensions from the calibration method 
described earlier can be used to provide a reference for 
determination of a side perspective view angle 177. The value D H 
Q and D B are known from the bottom view calibration. A side 
fy 15 perspective view image 210 comprising precision dots 211, 212 and 
p 213, corresponding to bottom view dots 201, 202 and 203 

respectively, of known spacing and dimensions D s and D h from the 
calibration method described earlier, can be used. to determine 
the side view perspective angle . The ratio of (D h / D H ) from the 
20 bottom image 2 00 and the side perspective image 210 can be used 
in the bottom view to calibrate D B in the same units as the side 
perspective view as follows: 

D Bc ai = D B (D h /D H ) 

Substituting into the equation for the side perspective view 
25 angle 177 described earlier yields: 
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9 = arcsin 



( n ^ 
u s 



= arcsin 



D B j 
6 = arcsin 



D 



s 



V Bcal J 



f DsDh A 
V DbDh j 



Figure 3A shows the apparatus of the invention for a three 
dimensional inspection of the balls of a ball grid array. The 
apparatus of the invention includes a part 70 to be inspected. 
™ The apparatus further includes a camera 10 with a lens 11, 
' y 10 located below the central area of part 70, to receive a bottom 
! fj image 80, described in conjunction with Figure 3B, of part 70. 
W The camera 10 is connected to a frame grabber board 12 to receive 

y 

lU the image 80. The frame grabber board 12 provides an image data 
Q output to a processor 13 to perform a two dimensional inspection 

ry 15 as described in conjunction with Figure 3A. The processor 13 may 
O store an image in memory 14. The apparatus of the invention 

\sss 

obtains an image of a pair of side perspective views with a 
camera 15 and a lens 16. The camera 15 is located to receive an 
image 90, comprising a pair of side perspective views, described 

20 in conjunction with Figure 3B and utilizing fixed optical 

elements 30, 32 and 38 for a first side perspective view and 
fixed optical elements 34, 36 and 38 for a second side 
perspective view. In one embodiment of the invention, the 
apparatus may contain a nonlinear optical element 39 to magnify 

25 the side perspective image 60 in one dimension as shown in Figure 
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8B. In another embodiment of the invention optical element 38 
may be the nonlinear element. The fixed optical elements 30, 32, 
34, 36 and 38 may be mirrors or prisms. As will be appreciated 
by those skilled in the art additional optical elements may be 
5 incorporated without deviating from the spirit and scope of the 
invention. The camera 15 is connected to a frame grabber board 
17 to receive the image 90. The frame grabber board 17 provides 
an image data output to a processor 13 to calculate the Z 
position of the balls, described in conjunction with Figure 3B. 

If* 10 The processor 13 may store an image in memory 14. 

: F= Figures 3B1, 3B2 and 3B3 show an example image 80 from 

|U camera 10 and an example image 90 from camera 15 acquired by the 

III 

fll system. The image 80 shows the bottom view of the balls located 
O on the bottom surface of a part 70. The image 90 shows two side 
ry 15 view perspectives of the balls located on part 70. A first side 
□ perspective view in image 90 contains images of balls 91 and is 
obtained by the reflection of the image of the part 70 off of 
fixed optical elements 30, 32 and 38 into camera 15. A second 
side perspective view in image 90 contains images of balls 92 and 
20 is obtained by the reflection of the image of the part 70 off of 
fixed optical elements 34, 36 and 38 into camera 15. Optical 
element 36 is positioned to adjust the optical path length of a 
second side perspective view to equal the optical path length of 
a first side perspective view. In one embodiment of the 
25 invention, the maximum depth of focus of a side perspective view 



just includes an area of the part including the center row of 
balls. This allows for a fixed focus system to inspect larger 
parts, with one perspective view imaging at least half of the 
part and the second perspective view imaging at least the other 
half of the part. Those skilled in the art will realize that any 
number of perspective views can be utilized by the invention. In 
another embodiment of the invention, all of the balls are in 
focus from both side perspective views resulting in two 
perspective views for each ball. This permits two Z calculations 
for each ball as shown in conjunction with Figures 10A and 10B. 

Figure 4 shows a flow diagram for the three dimensional 
inspection of balls on a ball grid array. The method starts in 
step 141 by providing a part 70 having balls 71 facing down. The 
method in step 142 provides a camera 10 located beneath the part 
70 to receive an image 80. In step 143 a frame .grabber 12 is 
provided to receive the image 80 from camera 10. In step 144, 
fixed optical elements are provided for obtaining two side 
perspective views of the part 70. A first optical path is 
provided by optical elements 30, 32 and 38. A second optical 
path is provided by optical elements 34, 36 and 38. A second 
camera 15 receives an image 90 of two side perspective views in 
step 145. In step 146 a second frame grabber board 17 is 
provided to receive the image 90 from camera 15. A processor 13 
sends a command to frame grabbers 12 and 17 to acquire images 80 
and 90 comprising pixel values from cameras 10 and 15. The 
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method then proceeds to step 147 and processes the pixel values 
with a processor 13 to obtain three dimensional data about part 
70. 

The invention contemplates the inspection of parts that have 
5 ball shaped leads whether or not packaged as a ball grid array. 
The invention also contemplates inspection of leads that present 
a generally curvilinear profile to an image sensor. 

Figures 5A and 5B together show a flow chart of the three 
dimensional inspection method of the invention. The process 
% Q 10 begins in step 151 by waiting for an inspection signal. When the 
45 signal changes state, the system initiates the inspection. The 

SSSB 

nj processor 13 sends a command to frame grabber boards 12 and 17 to 

:i i 

m acquire images 80 and 90 respectively of part 70 having balls 71. 
p In step 152, camera 10 captures an image 80 comprising pixel 

^ 15 values and camera 15 captures an image 90 comprising pixel values 
p{ and the processor stores the images in memory 14. The images 
|saB comprise information from both a bottom view and two side 

perspective views as shown in Figure 3B. In step 153, the 
inspection system sends a signal to a part handler shown in 
20 Figure 9 to allow the part handler to move the part out of the 
inspection area and allows the next part to be moved into the 
inspection area. The handler may proceed with part placement 
while the inspection system processes the stored image data. 
The inspection system processes the pixel values of the 
25 stored image 80 in step 154 to find a rotation, and X placement 
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and Y placement of the part relative to the world X and Y 

coordinates. The processor determines these placement values 

finding points on four sides of the body of the part. In step 

155, the processor employs a part definition file that contains 

5 values for an ideal part. 

By using the measurement values from the part definition 

file and the placement values determined in step 154, the 

processor calculates an expected position for each ball of the 

^ part for the bottom view contained in image 80. The processor 

*S 10 employs a search procedure on the image data to locate the balls 
08 

s P 81 in image 80. The processor then determines each ball's center 

Py location and diameter in pixel values using grayscale blob 

IV techniques as described with respect to Figure 7A. The results 

Si 

O are stored in memory 14. 

j|| 15 The processor proceeds in step 156 to calculate an expected 

r! position of the center of each ball in both side perspective 

views in image 90 using the known position of each side view from 
calibration. . The processor employs a subpixel edge detection 
method described with respect to Figure 7B to locate a reference 
20 point on each ball in step 157. The results are stored in memory 
14. 

Now refer to Figure 5B. In step 158 the processor converts 
the stored pixel values from steps 154 and 157 into world 
locations by using pixel values and parameters determined during 
25 calibration. The world locations represent physical locations of 
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the balls with respect to the world coordinates defined during 
calibration. 

In step 159 the Z height of each ball is calculated in world 
coordinates in pixel values. The method proceeds by combining 
5 the location of the center of a ball from the bottom view 80 with 
the reference point of the same ball from a side perspective view 
in image 90 as described in Figures 6A and 6B. The processor 
then converts the world values to part values using the 
calculated part rotation, and X placement and Y placement in step 

IsasF 

© 10 160 to define part coordinates for the ideal part. The part 

IB 

=P values represent physical dimensions of the balls such as ball 

lU diameter, ball center location in X part and Y part coordinates 

111 

IU and ball height in Z world coordinates. 

El 

q In step 161 these part values are compared to the ideal 

iy 15 values defined in the part file to calculate the deviation of 
each ball center from its ideal location. In one example 

S-JL, 

1 . embodiment of the invention the deviation values may include ball 

diameter in several orientations with respect to the X and Y part 
coordinates, ball center in the X direction, Y direction and 

20 radial direction, ball pitch in the X direction and Y direction 
and missing and deformed balls. The Z world data can be used to 
define a seating plane, using well known mathematical formulas, 
from which the Z dimension of the balls with respect to the 
seating plane can be calculated. Those skilled in the art will 

25 recognize that there are several possible definitions for seating 




25 

planes from the data that may be used without deviating from the 
spirit and scope of the invention. 

In step 162 the results of step 161 are compared to 
predetermined thresholds with respect to the ideal part as 
5 defined in the part file to provide an electronic ball inspection 
result. In one embodiment the predetermined tolerance values 
include pass tolerance values and fail tolerance values from 
industry standards. If the measurement values are less than or 
equal to the pass tolerance values, the processor assigns a pass 
•0 10 result for the part. 'If the measurement values exceed the fail 
s P tolerance values, the processor assigns a fail result for the 
III part. If the measurement values are greater than the pass 

l , s 

S3, S 

III tolerance values, but less than or not equal to the fail 

O tolerance values, the processor designates the part to be 

„ p 

|1j 15 reworked. The processor reports the inspection result for the 

part in step 163, completing part inspection. The process then 
l ~~ returns to step 151 to await the next inspection signal. 

Figures 6A and 6B show an example ball of a ball grid array 
and associated geometry used in a method of the invention for 
20 determining the Z position of the ball. The method determines the 
Z position of a ball with respect to the world coordinates 
defined during calibration. Using parameters determined from the 
calibration procedure as shown in Figures 2B and 2D to define a 
world coordinate system for the bottom view and the two side 
25 perspective views, comprising world coordinate plane 250 with 
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world coordinate origin 251 and world coordinate axis X 252, Y 
253 and Z 254 shown in Figure 6A, and a pair of images 80 and 90 
as shown in Figure 3B, the processor computes a three dimensional 
location . 

5 Now refer to Figure 6A. The processor locates a point 258 

on the world plane 250 determined by a bottom view ray 255 
passing through the center 257 of a ball 71 on a part 70, The 
processor locates a side perspective view point 2 60 on the world 
plane 250 determined by a side perspective view ray 256 

10 intersecting a ball reference point 259 on ball 71 and 

intersecting the bottom view ray 255 at a virtual point 2 61. Ray 
256 intersects the world plane 250 at an angle 262 determined by 
the reflection of ray 256 off of the back surface 263 of prism 



Q 30. The value of angle 262 was determined during the calibration 

fy 15 procedure . 



Now refer to Figure 6B. The distance Li is calculated by the 
processor as the difference between world point 258, defined by 
the intersection of ray 255 with the Z=0 world plane 250, and 
world point 260, defined by the intersection of ray 256 and the 
20 Z=0 world plane 250. The value Z is defined as the distance 

between world point 261 and 258 and is related to Li as follows: 



Z 

tan 0 j = 

L i 



Z = z i tan Oi 



Z can be computed by processor 13 since the angle 2 62 is 
known from calibration. The offset E 265 is the difference 
between the virtual point 2 61 defined by the intersection of ray 
255 and ray 256 and the crown of ball 71 at point 264, defined by 
the intersection of ray 255 with the crown of ball 71, and can be 
calculated from the knowledge of the angle 2 62 and the ideal 
dimensions of the ball 71. The final value of Z for ball 71 is: 

ZFinal = Z ~ E 

Figure 7A shows one example of an image used in the 
grayscale blob method of the invention. The image processing 
method finds the location and dimensions of a ball 71 from a 
bottom image 80. From the expected position of a ball 71, a 
region of interest in image 80 is defined as (XI, Yl) by (X2,Y2) . 

The width and height of the region of interest are large enough 
to allow for positioning tolerances of part 70 for inspection. 
Due to the design of the lighting for the bottom view, the 
spherical shape of balls 71 of part 70 present a donut shaped 
image where the region 281, including the perimeter of the ball 
71, comprises camera pixels of higher grayscale values and where 
the central region 282 comprises camera pixels of lower grayscale 
values. The remainder 283 of the region of interest 280 
comprises camera pixels of lower grayscale values. 

In one embodiment of the invention the processor 13 
implements image processing functions written in the C 



# # 



28 

programming language. 

The C language function "FindBlobCenter", as described 
below, is called to find the approximate center of the ball 71 by 
finding the average position of pixels that are greater than a 
5 known threshold value. The exact center of the ball 71 can be 
found by calling the C language function " FindBallCenter" which 
also returns an X world and Y world coordinate. Figure 7B shows 
one example of an image used with the method of the invention to 
perform a subpixel measurement of the ball reference point. The 
10 method of the invention finds a reference point on a ball 71 in 
= E an image 90 of a side perspective view as shown in Figure 3B. 
jV From the expected position of a ball 71, a region of interest 290 

y 

IU in image 80 is defined as (X3, Y3) by (X4,Y4). The width and 
O height of the region of interest are large enough to allow for 

Hi 15 positioning tolerances of part 70 for inspection. Due to the 

''■Jj / 

p design of the lighting for a side perspective view, the spherical 
M shape of balls 71 of part 70 present a crescent shaped image 291 

comprising camera pixels of higher grayscale values and where the 
remainder 293 of the region of interest 290 comprises camera 
20 pixels of lower grayscale values. 

The C language function "FindBlobCenter" is called to 
compute the approximate center of the crescent image 291 by 
finding the average position of pixels that are greater than a 
known threshold value. Using the coordinates of the approximate 
25 center of the crescent image 291, the C language function 
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"FindCrescentTop" is called to determine the camera pixel, or 
seed pixel 292 representing the highest edge on the top of the 
crescent. The camera pixel coordinates of the seed pixel are 
used as the coordinates of a region of interest for determining 
the subpixel location of the side perspective ball reference 
point . 

One example of grayscale blob analysis and reference point 
determination implemented in the C language is presented as 
follows : 



| ///////////// ////y ////////////////////////// ////////////////// 

.£ // FindBlobCenter - finds the X,Y center of the pixels that have 
III a value greater than THRESHOLD in the region (xl,yl) to (x2,y2) 

////////////////////////////////////////////////////////////// 
15 long FindBlobCenter (int xl,int yl,int x2,int y2, 

double* pX, double* pY) 



:» s 



int x,y; 

long Found = 0; 

20 long SumX = 0; 

long SumY = 0; 



for (x=xl ; x<=x2 ; x++ ) 
{ 

25 for (y=yl;y<=y2;y++) 

{ 

if (Pixel [x] [y] > THRESHOLD) 
{ 

SumX -i-= x; 
30 SumY += y; 

Found ++; 

} 

} 

} 



if (Found > 0) 
{ 

*pX = (double) SumX / (double) Found; 
*pY = (double) SumY / (double) Found; 
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return Found; 



////////////////////////////////////////////////////////////// 
// FindBallCenter - finds the X,Y center of the a BGA ball 
// using the grayscale values 

10 ////////////////////////////////////////////////////////////// 
long FindBallCenter (int xl,int yl,int x2,int y2, 

double* pX, double* pY) 

{ 

int x,y; 
15 long Found = 0 

long Total = 0 
long SumX = 0 
long SumY = 0 



20 



•!p 
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for (x=xl;x<=x2;x++) 
{ 

for (y=yl;y<=y2;y++) 



} 



{ 



if (Pixel [x] [y] > THRESHOLD) 
{ 

SumX += x*Pixel [x] [y] ; 
SumY += y*Pixel [x] [y] ; 
Total += Pixel [x] [y] ; 
Found ++; 



35 



if 

{ 



(Found > 0) 



*pX 
*pY 



= (double) SumX / 
= (double) SumY / 



(double) Total; 
(double) Total; 



40 



} 



return Found; 



//////////////////////////////////////////////////////•//////// 
// FindCrescentTop - finds the X,Y top position of a BGA crescent 
45 ////////////////////////////////////////////////////////////// 
void FindCrescentTop (int CenterX, int CenterY, int Diameter, 

int* pX,int* pY) 

{ 

int x, y, Edge, Max, TopX, Top Y; 



int xl = CenterX - Diamter / 2; 

int x2 = CenterX + Diamter / 2; 

int yl = CenterY - Diamter / 2; 

int y2 = CenterY; 

*pY = 9999; 

for ( x=x 1 ; x<=x2 ; x+ + ) 
{ 

Max = -9999; 

for ( y=y l ; y<=y2 ; y++ ) 

{ 

Edge = Pixel [x] [y] - Pixel [x] [y-1] ; 

if (Edge > Max) 

{ 

Max = Edge; 
TopY = y; 
TopX = x; 

} 

} 

if (TopY < *pY) 
{ 

*pX = TopX; 
*pY = TopY; 

} 

} 

(c) 1997 Scanner Technologies Inc. 

Figure 8A shows a side perspective image of the calibration 
pattern magnified in one dimension. Figure 8A shows a side 
perspective image 300 of a reticle calibration pattern where the 
space 303 between dot 301 and dot 302 is magnified, increasing 
the number of lower value grayscale pixels when compared to a non 
magnified image. 

Figure 8B shows a side perspective image of the balls on a 
BGA, magnified in one dimension. In Figure 8B a side perspective 
image 310 of two views are shown where the space 313 between ball 
image 311 and ball image 312 is magnified, increasing the number 
of lower value grayscale pixels when compared to a non magnified 
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image. The increased number of lower grayscale value pixels 
allows for the successful application of the subpixel algorithm. 

In another embodiment of the invention, the method and 
apparatus disclosed herein is a method and apparatus for 
5 calibrating the system by placing a pattern of calibration dots 
of known spacing and dimensions on the bottom plane of a 
calibration reticle and for providing for two side perspective 
views of each ball for the three dimensional inspection of parts. 
in From the precision dots the missing state values of the system 

;S 10 are determined allowing for three dimensional inspection of balls 
on BGA devices or balls on wafers or balls on die. 

sssa 

Hi Figure 9 shows an example apparatus for presenting a BGA to 

rtJ the system for inspection. An overhead light reflective diffuser 

□ 5 includes a vacuum cup assembly 6. The vacuum cup assembly may 
Ri 15 attach to a BGA part 70 having balls 71 and suspend the BGA part 

□ 70 below the overhead light reflective diffuser 5. 

Figures 10A and 10B show an example ball on a ball grid 
array and associated geometry for use with the method of the 
invention for determining the Z position of a ball with respect 

20 to the world coordinates defined during calibration, using two 
perspective views for each ball. Using parameters determined 
from the calibration procedure as shown in Figures 2B and 2D to 
define a world coordinate system for the bottom view and the two 
side perspective views, comprising world coordinate plane 700 

25 with world coordinate origin 701 and world coordinate axis X 702, 
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Y 703 and Z 704 shown in Figure 10A and Figure 10B, and a pair of 
images 80 and 90 as shown in Figure 3B, the processor computes a 
three dimensional location. 

Now refer to Figure 10A. The processor locates a point 709 
5 on the world plane 700 determined by a bottom view ray 705 
passing through the center 708 of a ball 717. The processor 
locates a first side perspective view point 711 on the world 
plane 700 determined by a side view ray 706 intersecting a ball 
..^ reference point 710 on ball 717 and intersecting the bottom view 
! J* 10 ray 705 at a virtual point 714. Ray 706 intersects the world 
T plane 700 at an angle 715 determined by the reflection of ray 706 
off of the back surface of prism 30. The value of angle 715 was 

W 

Til determined during the calibration procedure. The processor 

V. 

P locates a second side perspective view point 713 on the world 

[II 15 plane 700 determined by a side view ray 707 intersecting a ball 
O reference point 712 on ball 717 and intersecting the bottom view 

ray 705 at a virtual point 718. Ray 707 intersects the world 
plane 700 at an angle 716 determined by the reflection of ray 707 
off of the back surface of prism 34. The value of angle 716 was 
20 determined during the calibration procedure. 

Now refer to Figure 10B. The distance Li is calculated by 
the processor as the distance between world point 709 and world 
point 711. The distance L 2 is calculated by the processor as the 
distance between world point 713 and world point 709. The value 
25 Zi is defined as the distance between world point 714 and 709 and 



is related to Li as follows: 



tan 6i = — 
Li 

Zi = Li tan 0 } 



The value Z 2 is defined as the distance between world point 
718 and 709 and is related to L 2 as follows: 



tan 0 2 ~ 22 



L 2 

L 2 tan Q 2 10 



The average of Zi and Z 2 are calculated and used as the 
value for Z of the ball. This method is more repeatable and 
accurate than methods that use only one perspective view per 
ball. 

In still another embodiment of the invention, the method and 
apparatus disclosed herein is a method and apparatus for 
calibrating the system by placing a pattern of calibration dots 
of known spacing and dimensions on the bottom plane of a 
calibration reticle and for providing a single side perspective 
view for the three dimensional inspection of parts. From the 
precision dots the missing state values of t^|e system are 
determined allowing for three dimensional inspection of balls on 
BGA devices or balls on wafers or balls on die. 
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Figure 11A shows the apparatus of the invention for system 
calibration, utilizing a single side perspective view. The method 
and apparatus for calibration of the bottom view is identical to 
the method and apparatus described earlier in Figure 2A and 2B 
5 for the two side perspective views method. The apparatus for an 
image of a side perspective view includes a camera 15 with a lens 
18 and a calibration reticle 20. The camera 15 is located to 
receive an image 64 of a side perspective view comprising dots 
65, described in conjunction with Figure 11B, and utilizing fixed 
jO 10 optical elements 40 and 42. The fixed optical element 40 may be a 
■£» mirror or prism. The fixed optical element 42 is a nonlinear 
JU element that magnifies the image in one direction. In another 
ill embodiment fixed optical element 40 may be this nonlinear 
□ element. As will be appreciated by those skilled in the art 
ifj 15 additional optical elements may be incorporated. The camera 15 is 
*3 connected to a frame grabber board 17 to receive the image 64. 

The frame grabber board 17 provides an image data output to a 
processor 13 to perform a two dimensional inspection as described 
in conjunction with Figure 2B. The processor 13 may store an 
20 image in memory 14. 

Figures 11B1, 11B2 and 11B3 show an example calibration 
pattern and example images of a calibration pattern acquired by 
the system, utilizing a single side perspective view, of the 
invention. Figures 11B1, 11B2 and 11B3 show an example image 50 
25 from camera 10 and an example image 64 from camera 15 acquired by 



the system. The image 50 showing dots 52 acquired by camera 10 
includes a bottom view of the dot pattern 22, containing 
precision dots 24 of known dimensions and spacing, located on the 
bottom surface of the calibration reticle 20. The image 64 shows 
a side perspective view of the dot pattern 22, containing 
precision dots 24 of known dimensions and spacing, located on the 
bottom surface of the calibration reticle 20. A side perspective 
view in image 64 contains images of dots 65 and is obtained by 
the reflection of the image of the calibration reticle dot 
pattern 22 off of fixed optical element 40, passing through 
nonlinear element 42 and into camera 15. 

The side perspective calibration is identical to the method 
shown in Figure 2C except the fixed optical elements may have 
different properties . 

The determination of the state values for the side 
perspective view is identical to the method shown in Figure 2D 
except the fixed optical elements may be different and there is 
only one side perspective view. The principles and relationships 
shown in Figure 2E and Figure 2F apply. 

In still another embodiment employing a single side 
perspective view, the invention does not include the nonlinear 
element 42. 

Figure 12A shows the apparatus of the invention for ball 
inspection utilizing a single side perspective view. The 
apparatus of the invention includes a part 70 to be inspected. 
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The apparatus further includes a camera 10 with a lens 11, 
located below the central area of part 70, to receive a bottom 
image 80, described in conjunction with Figure 12B, of part 70. 
The camera 10 is connected to a frame grabber board 12 to receive 
5 the image 80. The frame grabber board 12 provides an image data 
output to a processor 13 to perform a two dimensional inspection 
as described in conjunction with Figure 12B. The processor 13 may 
store an image in memory 14. The apparatus for an image of a 
single side perspective view includes a camera 15 with a lens 18. 
*0 10 The camera 15 is located to receive an image 94, comprising a 

m 

:p single side perspective view, described in conjunction with 

rij Figure 12B and utilizing fixed optical element 40 and nonlinear, 

ftj fixed optical element 42, to magnify the side perspective view in 

□ one dimension. In another embodiment of the invention optical 

m 15 element 40 may be the nonlinear element. The fixed optical 

;5 element 40 may be a mirror or prism. As will be appreciated by 

jt * those skilled in the art additional optical elements may be 

incorporated. The camera 15 is connected to a frame grabber board 
17 to receive the image 94. The frame grabber board 17 provides 
20 an image data output to a processor 13 to calculate the Z 

position of the balls, described in conjunction with Figure 12B. 
The processor 13 may store an image in memory 14. 

Figures 12B1, 12B2 and 12B3 show an example ball grid array 
and example images of the ball grid array for three dimensional 
25 inspection, utilizing a single side perspective view. Figures 
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12B1, 12B2 and 12B3 show an example image 80 from camera 10 and 
an example image 94 from camera 15 acquired by the system. The 
image 80 shows the bottom view of the balls 71 located on the 
bottom surface of a part 70. The image 94 shows a side 
5 perspective view of the balls 71 located on part 70. The side 
perspective view in image 94 contains images of balls 95 and is 
obtained by the reflection of the image of the part 70 off of 
fixed optical element 4 0 and passing through the nonlinear fixed 
element 42 into camera 15. 

*fi io In an alternate embodiment of the invention, the system can 

IB 

<p be used to inspect other types of electronic parts in three 
ry dimensions, such as gullwing and J lead devices. By utilizing 

t . I! 
\&$ 

|1J only one camera and adding an additional set of prisms on the 
O reticle 400 these other devices may be inspected. The advantage 
iij 15 of being able to inspect different devices with the same system 
;-5 includes savings in cost, and floor space in the factory. 

Additionally this design allows more flexibility in production 
planning and resource management. 

Figure 13 shows the apparatus of the invention for the three 
20 dimensional inspection of ball grid array devices, gullwing 

devices and J lead devices. The apparatus described in Figure 13 
allows the inspection of BGA, gullwing and J lead devices all on 
the same system. The apparatus includes a part 402 to be 
inspected located over the central area of a transparent reticle 
25 400 with prisms 401 glued to the top surface to receive side 



perspective views of part 402. A gullwing and J lead inspection 
device 21 may be integrated into the ball grid array inspection 
device. One example embodiment of such a gullwing and J lead 
inspection device is the "UltraVim" scanner from Scanner 
Technologies of Minnetonka, Minnesota. The apparatus further 
includes a camera 10A with a lens 11A, located below the central 
area of part 402 and reticle 400 to receive a bottom view and 
side perspective views of part 402. The camera 10A is connected 
to a frame grabber board 12A to receive an image. The frame 
grabber board 12A provides an image data output to a processor 
13A to perform a three-dimensional inspection of part 402. The 
processor 13A may store an image in memory 14A. These components 
comprise the hardware of the gullwing and J lead inspection 
device 21 and are shared by the ball grid array inspection device 
as described herein. 

The UltraVim is described in United States Patent Number 
6,055,054 entitled THREE DIMENSIONAL INSPECTION SYSTEM by Beaty 
et al., issued April 25, 2000 which is incorporated in its 
entirely by reference thereto. 

Refer now to Figure 14. In still another embodiment of the 
invention, the system may use three cameras to image directly the 
bottom view and two side perspective views as shown in Figure 14. 

Figure 14 shows the apparatus of the invention for a three 
dimensional inspection of the balls of a BGA. The apparatus of 
the invention includes a part 70, with balls 71 to be inspected. 
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The apparatus further includes a camera 10 with a lens 11, 
located below the central area of part 70, to receive a bottom 
image 80, described in conjunction with Figure 12B, of part 70. 
The camera 10 is connected to a frame grabber board 12 to receive 
5 the image 80. The frame grabber board 12 provides an image data 
output to a processor 13 to perform a two dimensional inspection 
as described in conjunction with Figure 12B. The processor 13 
may store an image in memory 14. The apparatus for an image of a 
first side perspective view includes a camera 15 with a lens 19. 

13 

*S 10 The camera 15 is located to receive an image 94, comprising a 
13 

7 f single side perspective view, described in conjunction with 

i -Tee 

111 Figure 12B and utilizing fixed optical element 38, to magnify the 

y 

FU side perspective view in one dimension. The camera 15 is 

O connected to a frame grabber board 17 to receive the image 94. 
ry 15 The frame grabber board 17 provides an image data output to a 
13 processor 13 to calculate the Z. position of the balls, described 

in conjunction with Figure 12B. The processor 13 may store an 
image in memory 14. The apparatus for an image of a second side 
perspective view includes a camera 15 with a lens 19. The camera 
20 15 is located to receive an image similar to 94, comprising a 
single side perspective view, described in conjunction with 
Figure 12B and utilizing fixed optical element 38, to magnify the 
side perspective view in one dimension. The camera 15 is 
connected to a frame grabber board 17 to receive the image 
25 similar to 94. The frame grabber board 17 provides an image data 
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output to a processor 13 to calculate the Z position of the 
balls, described in conjunction with Figure 12B. The processor 
13 may store an image in memory 14, In another embodiment, the 
nonlinear fixed optical element 38 may be missing. In still 
5 another embodiment of the invention, only one side perspective 
view may be utilized. 

In another embodiment of the invention, the method and 
apparatus disclosed herein is a method and apparatus using a 
single camera for calibrating the system by placing a pattern of 
k D 10 calibration dots of known spacing and size on the bottom plane of 
•P a calibration reticle. From the precision dots the missing state 

ares 

RJ values of the system are determined allowing for three 

E - 3 

|1J dimensional inspection of balls on ball grid array devices, BGA 

Q devices or balls on wafers or balls on die. 

j|| 15 Refer now to Figure 15 which shows one example of the 

A apparatus of the invention configured with a calibration reticle 

1020 for use during calibration of the state values of the 
system. Calibration reticle 1020 is positioned to be viewed by 
camera 1008. Camera 1008 further comprises a lens 1006. Camera 
20 1008 receives a composite image of the calibration reticle 1020, 
one portion of the image, through mirror 1002 and an additional 
portion directly. A frame grabber 1010 receives image 
information from the camera 1008 and provides processor 1012 with 
image information of the calibration reticle 1020. The image 
25 information and other inspection information may be stored in 



memory 1014. The apparatus obtains an image 1024 showing a 
bottom view 102 6 containing an image 1031 of a precision dot 1030 
and a side view 1028 containing an image 1032 of precision dot 
1030 of the calibration reticle 1020 shown in Figure 16A. This 
image 1024 is shown in Figure 16B. To take the image 1024, the 
apparatus includes a means of illumination 1017, an overhead 
light diffuser 1015, the camera 1008, with lens 1006, and 
calibration reticle 1020 with a calibration pattern 1022 on the 
bottom surface of the calibration reticle 1020. A separate 
optical element 1002 is positioned below the calibration reticle 
1020 to provide an additional perspective or side view 1028 
containing an image of precision dot 1032 of the calibration 
reticle 1020. 

In one embodiment of the invention, the optical element 1002 
may comprise a prism. In another embodiment of the invention, 
the optical element 1002 may comprise a mirror. As will be 
appreciated by one skilled in the art, the invention will work 
with any number of side views. The calibration pattern 1021 on 
the reticle 1020 comprises precision dots 1022. The camera 1008 
is located below the central part of the calibration reticle 1020 
to receive an image 1024 described in conjunction with Figures 
16A and 16B. In one embodiment of the invention, the camera 1008 
comprises an image sensor. The image sensor may be a charged 
coupled device array. The camera 1008 is connected to a frame 
grabber board 1010 to receive the image 1024. The frame grabber 
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board 1010 provides an image data output to a processor 1012 to 
perform a three dimensional calibration as described in 
conjunction with Figure 16B. The principles and relationships 
shown in Figure 2E and 2F apply. The processor 1012 may store an 

5 image in memory 1014. 

Now refer to Figures 16A and 16B which show a calibration 
reticle 1020 having precision dots 1022. Figure 16B shows a 
composite image 1024 of the calibration reticle 1020. The bottom 
view 1026 shows precision dot 1030 with a first perspective view 

10 1031 and a side view 1028 shows precision dot 1030 with a second 
perspective view 1032. The system processor processes the 



Hj composite image 1024 according to the system of equations 

m 

described herein with the bottom view 1026 and side view 1028 



providing information for the solution of the system of 
15 equations. The principles and relationships shown in Figure 2E 
and Figure 2F apply. 

In another embodiment of the invention, the method and 
apparatus disclosed herein is a method and apparatus using a 
. single camera for a three dimensional inspection of balls on ball 
20 grid array devices, BGA/CSP devices or balls on wafers or balls 
on die. 

Refer now to Figure 17 which shows the apparatus of the 
invention configured with a part 104 0 to be inspected by the 
system. The apparatus obtains an image 1044 showing a bottom 
25 view 1046 containing an image of a ball 1050 and a side view 1048 
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containing an image of a ball 1052 of the part 1040. To take the 
image 1044, the apparatus includes a means of illumination 1017 
which may be a ring light, camera 1008 with a first optical 
element, lens 1006, and a part 1040 with a ball 1042 on the 
5 bottom surface. In one embodiment of the invention, the means of 
illumination 1017 lights the bottom surface of the part 1040 to 
allow imaging the perimeter of the part 1040. In another 
embodiment of the invention, overhead diffuser 1015 provides 
illumination for imaging of the perimeter of the part 1040. 

Q 

*fi io In one example, the means for illumination may comprise 

m 

& reflected light, the lens 1006 may comprise a plurality of lens 

fy elements, a pin hole lens or a telecentric lens, and the 
UJ 

!y processor 1012 may comprise a personal computer. Those skilled in 
□ the art will understand that the output of the sensor may be 
fy 15 transmitted directly to memory without the use of a frame 
i~5j grabber. 

^ A separate second optical element 1002 is positioned below 

the bottom plane of part 104 0 to provide an additional 
perspective or side view 1048 of the part 1040 containing an 
20 image of the ball 1052 of the part 1040. In one embodiment of 

the invention, the optical element 1002 may comprise a prism. In 
another embodiment of the invention, the optical element 1002 may 
comprise a mirror. As will be appreciated by one skilled in the 
art, the invention will work with any number of side views. The 
25 camera 1008 is located below the central part of the part 1040 to 




45 

receive an image 1044 described in conjunction with Figure 18A 
and 18B. In one embodiment of the invention, the camera 1008 
comprises an image sensor. The image sensor may be a charged 
coupled device array or a complementary metal oxide 
5 semiconductor device array. The camera 1008 is connected to a 
frame grabber board 1010 to receive the image 1044. The frame 
grabber board 1010 provides an image data output to a processor 
1012 to perform a three dimensional inspection of the part 1040 
as described in conjunction with Figures 18A and 18B. The 
*S 10 principles and relationships shown in Figure 6A and Figure 6B 

i:o 

: fc apply. The processor 1012 may store an image in memory 1014. 
W Now refer to Figures 18A and 18B which shows a ball grid 

! 3 3 

iH array 1040 having balls 1042. Figure 18B shows a composite image 

0 1044 of the ball grid array 1040. The bottom view 1046 shows 

flf 15 ball 1050 with a first perspective view 1051 and a side view 1048 

q shows ball 1050 with crescent shape 1052 with a second 

! ~ 

perspective view. The system processor processes the composite 
image 1044 according to the system of equations described herein 
with the bottom view 1046 and side view 1048 providing 
20 information for the solution of the system of equations. The 
principles and relationships shown in Figure 6A and Figure 6B 
apply. 

The invention has been described herein in considerable 
detail in order to comply with the Patent Statutes and to provide 
25 those skilled in the art with the information needed to apply the 



novel principles and to construct and use such specialized 
components as are required. However, it is to be understood that 
the invention can be carried out by specifically different 
equipment and devices, and that various modifications, both as to 
the equipment details and operating procedures, can be 
accomplished without departing from the scope of the invention 

itself . 

What is claimed is: 



